Efficient Imbalanced Multimedia Concept Retrieval by Deep Learning on Spark Clusters

The classification of imbalanced datasets has recently attracted significant attention due to its implications in several real-world use cases. In such scenarios, the datasets have skewed class distributions while very few data instances are associated with certain classes. The classifiers developed on such datasets tend to favor the majority classes and are biased against the minority class. Despite extensive research interests, imbalanced data classification still remains a challenge in data mining research, especially for multimedia data. Our attempt to overcome this hurdle is to develop a convolutional neural network (CNN) based deep learning solution integrated with a bootstrapping technique. Considering the fact that convolutional neural networks are very computationally expensive coupled with big training datasets, we propose to extract features from pre-trained convolutional neural network models and feed those features to another full connected neutral network. Spark implementation shows promising performance of our model in handling big datasets with respect to feasibility and scalability.